Gallery-free Methods for Detecting and Recognizing People and Groups of Interest

by Jeremiah R. Barr The 2013 Boston Marathon investigation presented an ample opportunity to apply advanced face recognition technology. Unfortunately, the crowded scenes proved too difficult for existing facial biometrics methods and databases. In this dissertation, we identify some of the limitations of this technology and present approaches for overcoming these deficiencies. These approaches underlie a suite of novel algorithms for detecting faces in unconstrained imagery; clustering images of faces into identity groups; finding people who appear frequently across a collection of scenes; and detecting groups of individuals who often appear together. Face detection typically serves as the first step in a facial analysis pipeline. We elucidate the difficulties encountered on uncontrolled imagery by a simple yet strong-performing face detector. Although pose has traditionally been the focus of much of the research on face detection, we show that occlusion and blur have at least as significant of an impact for data collected from natural scenes. The results also indicate that classifiers with distinct error modes can result from blur-based perturbations of a detector training set, enabling the successful application of fusion techniques. Identity clustering plays a similar role to face detection insofar as it can drive automatic ”pattern of life” analyses. We introduce an active clustering scheme, the Framework for Active Clustering with Ensembles (FACE), with an emphasis on clustering face